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Abstract: Under some mild Markov assumptions it is shown that the problem of designing optimal sequential tests 
for two simple hypotheses can be formulated as a linear program. This result is derived by investigating the Lagran- 
gian dual of the sequential testing problem, which is an unconstrained optimal stopping problem depending on two 
unknown Lagrangian multipliers. It is shown that the derivative of the optimal cost function, with respect to these mul¬ 
tipliers, coincides with the error probabilities of the corresponding sequential test. This property is used to formulate 
an optimization problem that is jointly linear in the cost function and the Lagrangian multipliers and can be solved for 
both with off-the-shelf algorithms. To illustrate the procedure, optimal sequential tests for Gaussian random sequences 
with different dependency stmctures are derived, including the Gaussian AR(1) process. 
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1. Introduction 


The treatment of sequential analysis, in general, and sequential hypothesis testing, in particular, can be 


roughly divided into two main strands. The first one was started by Wald (1947 1 in his pioneering work 


on sequential analysis. Without being aware of it, Wald used fundamental properties of martingales to 
establish bounds on the error probabilities and the expected run-length of sequential tests between two 
simple hypotheses. In the decades after Wald’s initial publication, research on random processes made 
significant progress. Theories of martingales, renewal, and Levy processes emerged and became powerful 
tools in sequential analysis ( Lai[ 1977[ 2009[ Buonaguidi] 20131. They allowed for elegant derivations 
of asymptotically optimal procedures on the one hand, and bounds or approximations on non-asymptotic 
performance measures on the other. Following Wald’s footsteps, many of the theoretical findings obtained 
this way resulted in elementary guidelines for the design of sequential tests that were readily applicable 
in fields as diverse as survival analysis ( |Sellke[ 19831, radar sensing ( [Marcus[ 1962| |, and image processing 
( Basseville] 19811. For the design of strictly optimal procedures, however, a conceptually different approach 
to the problem proved to be more successful. 


The second strand of sequential analysis was started by Bellman (1954 1 with his publications on dynamic 
programming. Coming from a background in physics and computer science. Bellman sought to develop a 
theory “to treat the mathematical problems arising from the study of various multi-stage decision processes”, 
which can be found in “virtually every phase of modern life, from the planning of industrial production lines 
to the scheduling of patients at a medical clinic [...]” (Bellman| [1954 1. Sequential hypothesis testing. 
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however, had not been added to this list before the 1960s, when [Chow and Robbins] ( [1963| l embedded a 
general theory of optimal stopping in the dynamic programming framework. Treating a sequential test not 
as a single threshold crossing problem, but as a sequence of individual decisions to either stop or continue the 
test, added significantly to the general insight into sequential inference problems and made the theoretical 
derivation of strictly optimal methods possible ( Chow et’ah] 1971] Novikov[ 20091. 

Naturally, both approaches have coalesced over the years and nowadays sequential testing is usually 
treated as an optimal stopping problem that exploits certain stochastic properties of the underlying random 
process—see Buonaguidi ( |2013| ) for a recent example. However, to the present day, the use of dynamic 
programming methods to design sequential tests is uncommon in practice. Most often, optimal stopping 
theory is rather used to prove the existence of an optimal rule or to derive its general form. Wald’s sequential 
probability ratio test is a classic example. While optimal stopping theory provides a seamless and elegant 
way to show its optimality in the i.i.d. case ( Shiryaev]|1978| ), obtaining the exact values of the thresholds is 
much harder a task and requires considerable computational effort. 

The reason for this high computational cost is twofold: First, the Bellman equation of the optimal 
stopping problem itself needs to be solved. Traditionally, and owing to its roots in dynamic programming, 
iterative backward recursion procedures ( Bellman} 1954) are used for this purpose. Under certain conditions, 
more efficient methods are applicable ( jHelmes 20021, including a number of linear programming (LP) 
techniques ( Manne] I960] Denardo[ 1979] Rdhl[|20()T] ). The second issue that arises when deriving optimal 
strategies is that the sequential testing problem can not be tackled directly, but has to be reformulated in order 
to fit the optimal stopping framework. This involves the introduction of two cost coefficients that determine 
the price for making an error of either type. The optimal stopping strategy depends on the choice of these 
coefficients. The design of sequential tests in an optimal stopping framework, therefore, requires a second, 
outer optimization over the unknown cost coefficients. These intricacies limit the design and application of 
strictly optimal decision strategies. 

The aim of this paper is to overcome these issues, by unifying the stopping problem and the problem of 
choosing the right cost coefficients. The formulation of sequential testing as an optimal stopping problem 
is briefly addressed in Secfion [^ where also fhe Bellman equation fhaf characferizes ifs solufion is slated. 
In Secfion Ihis solufion is sludied in delail and a conneclion belween ifs derivalive wifh respecl fo fhe 
cosl coefficienfs and fhe error probabilifies of fhe sequenlial fesf is derived. Based on Ihese resulls, fhe 
original sequenlial lesling problem is Ihen addressed in Section]^ where if is shown, under cerlain Markov 
reslriclions, lhal if can be formulaled as a joinlly linear opfimizafion over fhe cosl coefficienfs and fhe oplimal 
slopping slrafegy. This conslilufes fhe main Iheorelical confribulion of fhe work. Since linear programming 
is rarely ever considered in fhe confexl of sequenlial inference, esfablishing Ibis connection can be seen as 
resull in ifs own righl. In addifion, fhe main Iheorem is of high pracfical relevance since if allows a large class 
of sequential lesling problems lo be Irealed wilhin one consislenl framework and solved wifh off-lhe-shelf 
linear programming algorithms. This procedure is demonstrated in Section]^ where optimal sequential tests 
are designed for i.i.d. observations, an observable Markov chain and the Gaussian AR(1) process. 

A note on notation: M+ denotes the positive reals and the associated Borel cr-algebra. Random 
variables are denoted by upper-case letters, their realizations by the corresponding lower-case letters. All 
(in)equalities between tuples have to be read element-wise. 


2. Sequential Tests for Two Simple Hypotheses 

The problem of sequentially testing between two simple hypotheses, under different restrictions and as¬ 
sumptions, has been treated extensively in the literature; see, for example, Tartakovskyj (2014 1 and refer¬ 
ences therein. The purpose of this section is to give a summary of known results, as well as to present them 


in a form that facilitates the derivations in the subsequent sections. It closely follows Novikov (20091 in 
terms of argumentation and notation. 
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Let {Xn)n>i, with n G N, be a sequence of random variables in a metric state space {Ex,£x)^ defined 
on some filtered probability space (O, E, {Xn)n>o, P)- The two simple hypotheses are given by 

no:P = Po, 

Pi :P = Pi. 


Kolmogorov’s consistency theorem ( Kallenberg| [1997 1 states that P is uniquely defined by the marginal 
distributions of all subsequences Xi,, Xn, n > 1. Let these distributions be denoted Fq and P” under 
Po and Pi, respectively. The classic sequential testing problem is to design a test that guarantees certain 
error probabilities under Pq and Pi and minimizes the run-length of the test under some third measure P, 
corresponding to a sequence of marginal distributions (P”)„>i. In general, P can be chosen arbitrarily; 
common choices are P = Pq or P = Pi, which correspond to run-lengths minimizations under either 
hypothesis. 


2.1. Assumptions 


The framework proposed in this paper covers processes {Xn)n>i that satisfy the following three assump¬ 
tions. A bullet • is used to indicate that an assumption holds under P, Pq and Pi. 


1. (X„)„>i admits a time-homogeneous Markovian representation. This means that a sequence of suf¬ 
ficient statistics (0n)n>o ^ ^P^^e {Eq, £g) exists such that 

P,{Xn+l G P I Pn) = P.{Xn+l G P | 0„) =1 P.,e„(P) 

for all n > 0 and all P G ■ It is further assumed that a P-measurable function ^ : Ex x Eg ^ Eg 
exists such that 

0„ = e(Pn,0n-l)=:?e„_l(Pn) with 00 = 00- 

In words, the distribution of X^+i is completely specified by 0„, which can in turn be calculated 
recursively from 9n-i and the observation Xn- The initial value 9q is deterministic and given a priori. 
Random sequences admitting these properties cover a wide range of commonly used models, such as 
ARMA and ARCH models and general Markov chains. In order to avoid technical difficulties. Eg is 
assumed to be Borelian. 


2. The density functions /”,/^, and /f, corresponding to F^,Fq, and P”, exist with respect to some 
common product measure /r” = (8i • • • (8) p^ixn) and can, according to the first assumption, be 

written as 

n n 

f^{xi,. .. ,Xn) = Yl f,{xk\ 9 k-i) =■■ n 

k=l k=l 


for all n > 1. 


3. The Radon-Nikodym derivatives, or likelihood ratios. 


: E 


X 


with zf 


dP" 


TT (Xk) 

fdk-iixk) ’ 


i = 0,1 


are jointly continuous random variables. Note that this implies that Fq and P[* are dominated by 
F^ for all n > 1. The reasons for this particular choice of assumptions will become apparent in the 
course of the paper. 
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2.2. Constrained Problem Formulation 

Let the sequence ip = with ipn = ipn{xi, ■ ■ ■ ,Xn) G {0,1}, define the stopping rule of the 

sequential test. Here ipn = '^ denotes the decision to stop at time instant n and ipn = 0 denotes the decision 
to continue testing. Analogously, let p = i4’n)n>i’ with = (j)n{xi, ■ ■ ■, Xn) G {0,1}, be a sequence of 
decision rules, where 4>n = 3 indicates a decision for hypothesis j, given that the test stops at time n. It is 
shown later that the restriction of continuously distributed likelihood ratios avoids the need for randomized 
stopping and decision rules. 

Let r = r(V’) be the stopping time that corresponds to the stopping rule ip, i.e., 

T = min{n >0 : V’n = !}• 

The error probabilities of the first and second kind, ao and ai, are given by 

ao{ip,(p) = Po[<pT = 1], 
ai{ip,(p) = Pi[(pT = 0]. 

The sequential testing problem is to solve 

min E[t( ip)] s.t. ai{ip,(p) < ji, f = 0,l, (2.1) 

where 7 = ( 70 , 71 ) G ( 0 , 1 )^ := ( 0 , 1 ) x ( 0 , 1 ) are bounds on the error probabilities, or target error 
probabilities, and the expected value is taken with respect to P. The solution of ( |2.1| ) is denoted {ipp, pp). 


2.3. Unconstrained Problem Formulation 


The commonly used procedure to solve ( |2.1[ ), in an optimal stopping framework, is to reformulate the 
constrained problem as an unconstrained cost minimization by showing that two constants Aq, Ai >0 exist 
such that the solution of 

min E[t{iP)] + AoQ!o(V’, 4>) + Aiq;i(V', P), (2.2) 

in the following denoted p\), coincides with the solution of ( |2.1| ), i.e.. 


VtE (0,1)' 


3A G 


ip;,p;) = iPl,Pl). 


This method has been used early on to prove optimality of the sequential probability ratio test for i.i.d. 
observations ( Wald[ 19481; a general proof can be found in Novikov ( |2009 1. The cost coefficients Aq and 
Ai in (^1 act as Lagrangian multipliers. However, the question how to choose them in order to meet the 
constraints on the error probabilities in p.l| ) has received little to no attention in the literature. It is discussed 
in detail in the next section. 

For now, A is assumed to be given and fixed. Following fhe usual line of argumenfs, fhe second parf of 
fhe objective funcfion in ( |2.2| ) can be wriffen as 

Xoao{p,p) + Xiai{p,p) = ^ (AoF^o[^nl{<^„=i}] + XiEi[pn'i-{4,,,=o}]) 

n>l 

[ V'n(Ao/o l{0„=i} + ^i/ri{</>n=o}) d/r”. 

\ 1 


n>l ' 


The cosf minimizing decision rules p*^ are hence given by 

(Pl,n = l{Ao/o"<Ai/i"}, n>l, 


(2.3) 
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where 


{Ao/o < Al/ri := {(xi,...,Xn) G Ex : Aq/q (xi, . . . , Xn) < Ai/f(xi,...,x„)}. 

This shorthand notation is used for subsets of the state space throughout the paper. The choice in ( |2.3| ) that 
the ambiguous event {XofQ = Ai/f} leads to a decision for 1-Lq is arbitrary since P({Ao/^ = Ai/f}) = 0 
for all n > 1 by Assumption 3. 

The decision rule ( |2.3| ) corresponds to a classic likelihood ratio test with threshold Aq/Ai. Knowledge 
of the likelihood ratio /” / is therefore sufficient to decide for a hypothesis once the test has stopped. The 

optimal stopping strategy, however, requires additional information as will become apparent later on. 
Substituting p.3| ) into p.2[ ) yields 

^[r(V')] + Xoaoiip, 4>\) + Aiai(^/;, 0 ^) = ^ f V’nCn/” + min{Ao/o , Ai/r}) d/r” 

n>l 

= ^ / ipnin + min{AoZo , Ai^^)/” d^” 

n>l 

= (” + 5a(2:”))] , 

n>l 


where = (zq, zf) and 


gx{z) = min{Aozo, Aizi}. 


Problem (2.2 1 therefore reduces to the optimal stopping problem 


min 14(V’) ■='^E [ipn (n + 5 a(^”))] 

in * ^ 


(2.4) 


n>l 


with recursively defined state variables 


f'i _n-l fi,Sn-l 




fSn-l 


, f = 0, 1, 


and z^ = 1. The solution of (2^) is given in the following theorem. 

Theorem 2.1. Let X > 0 be given. Under the three assumptions stated above, the functional equation 

px{z,e) =min|5A(2) , 1 + j px ^fe{x) ’ 

has a unique solution px > 0 on (E, £) := (M^ x Eg.,B\ ® £g) and it holds that 

Px{lA,eo) = Vx{f*x). 


(2.5) 


A proof of theorem 2.1 is given in Appendix [A| By a change in measure, (^1 can alternatively be 
written as 


px{z,9) = m.m.^ygx{z) , 1 + j pAdiT^,6i|, 
where q : (z, 6) G £^} is a family of probability measures on {E, £) satisfying 

xD) = Fe Qx G Ex : ^ ^ ^ 


( 2 . 6 ) 


(2.7) 
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for all B G and D £ £q. Use of both formulations is made in the following. The notation H, 


z,6»’ 


i = 0,1, is used to refer to the families of distributions where F in ( |2.7| ) is replaced by Fj. Note that g\ is 
i/j, 0 -integrable for all A G and {z, 6) £ E since 


gx < I ^ozo^ dFe = Xqzq < oo. 


Therefore, p\ < gx is ^-integrable as well. 

The importance of Theorem |2.1| lies in the fact that it allows the optimal stopping region to be specified 
on the codomain of {z'^, On), which is time invariant, rather than the sequence of product spaces (F^)„>i 
corresponding to the raw observations. Let the stopping region Sx, its boundary dSx, and its complement 
5 a be defined by 


5a = |(z,6') G F : gx{z) < 1 + j pxdHz^e^ , 

dSx = ^{z,e) £ E : gx{z) = 1 + j pxdH^^g^ , 

^A = e F : gx{z) > 1 + j pxdH^^g^ . 


The connection between the optimal stopping policy ^|}* and the functional equation in Theorem |2.1| is stated 
in the following corollary. 

Corollary 2.1, Under the assumptions given above, the stopping policy that minimizes Vx in ^2.4^ fulfills 

4sx < 1px,n < 1cSaU95a Vn > 1. 


The corollary follows immediately from the the definition of px, cf. Novikov (20091. The time-invari- 
ance of the stopping strategy reflects the time-homogeneity of the Markov sequence underpinning the test. 
In Sectionit is further shown that under the given assumptions dSx is a F null set so that the relations in 
Corollary |2.1 [ hold with equality in an almost sure sense. 

Theorem [2T] and, in particular, the implicit definition of the cost function px in ( |2.5| ), provides the basis 
for formulating the sequential detection problem as a linear program. This formulation requires several 
properties of the cost function px, which are detailed in the following section. 


3. Properties of the Cost Function px 

In this section, some basic properties of px are derived and the close relationship between its derivatives 
with respect to A and the error probabilities of the corresponding sequential test are shown. Some technical 
issues need to be addressed first. 

Lemma 3.1. The sequence {p^n>o with p” = T^{gx) and T defined in ( |A.1| ), converges uniformly on E. 
The proof of Lemma [3T] is detailed in Appendix [B| 

Sequences of functions of the same structure as {pf)n>o, where the reth function is defined recursively 
via an integration over the previous function, are encountered repeatedly in this section. The same arguments 
as detailed in Appendix can be used to show that in such cases almost uniform convergence implies 
uniform convergence. Therefore, instead of giving a formal proof again, we refer to Lemma [3T] in what 
follows. 

The next lemma bounds the influence of the likelihood ratio on the associated cost. Qualitatively speak¬ 
ing, it states that px is monotonic and sublinear in 2 ;. 
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Lemma 3.2. For all a,zG and 6 G Eq, the cost function p\ as defined in p.5| ) satisfies 
min{ao,ai, < p\{aozo,aizi,6) < max{ao, ai, 1 }pa( 2 , 6»). 


Proof. We only prove the lower bound since the upper one can be shown analogously. Assume that the 
lemma holds for some p” and let a* = min{ao, oi, 1}- By induction it holds that 


p^'^^{aoZo,aiZi, 9 ) = mmi^gx{aoZo,aiZi), 1 + j px 

> min |a*P a( 2 ) , a* + a* J px 


= a*pl+\z,e). 


The induction basis is Pa(« o 2 o, ai-zi) > a*gx{z). 
guarantees that the bound holds for px as well. 


By 


Rudin 


(19871, uniform convergence of {p 


n>l 

□ 


Using the bounds in Lemma it is easy to show that the boundary dSx is indeed a P null set. 

Lemma 3.3. If the likelihood ratios z^ are continuous random variables for all n > 1, the boundary dSx 
of the optimal stopping region is a null set under P, i.e., 


H,,e{dSx) = 0 ^{z,9)gE. 

Proof. Lemma [3^ can be shown by contradiction. Assume that Hzp{dSx) > 0 for some (z, 9), i.e., assume 
that if (z”, 9n) = {z, 9), then there is a nonzero probability that the test hits dSx with the next update of 
the test statistic. Since all z"^ are assumed to be continuous random variables, this implies that an interval 
[az*, 2 ;*], a < 1, exist on which 


1 + 


PAdi7^,e = gx{z) 


for all 2 ; G [az*, z*]. However, by Lemma 3.2 


1 + / P\d-Haz*fi >1 + j apx 


> a (1 + /pA 

= a (^1 + j px 
= ag\{z*) = g\{az*), 


which contradicts the assumption. □ 

Lemma [T^ implies that the optimal sequential test is non-randomized since the cost minimizing decision 
is almost surely unambiguous. 

It is now possible to give expressions for the derivatives of px. 

Theorem 3.1. Let p\. denote the derivative of px with respect to Xi, z = 0,1. Further, define 


Sxi •— Sx n {XiZi < 
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The two Fredholm integral equations of the second kind 


rx,iz,e) = ZiHlg{SxJ + [_ rx,dH^^e, i = 0,l, 

J*Sx 


(3.1) 


have a unique solution r\ = rxf) on Sx and it holds that 


p'xAz,e) = < 


Zi, {z, 9 )€Sxi 

rxi{z,e), {z,d)eSx 


,0) {z,9)€Sxi_i- 

A proof of Theorem [3^ can be found in Appendix]^ 

Based on Theorem |3.1i the main result of this section can be shown, which is the connection between 
the cost coefficients A and the error probabilities of the corresponding cost-minimizing sequential test. 

Theorem 3.2. For p'^ _, as defined in Theorem 


3.1 


it is the case that 


p'xiiz, 9) = ZiPi[<p*x^., = I - ilz'^ = z,9o = 9], i = 0,1 
on E\ dSx and in particular 

p'xMA,^o) = ai{(j)*x,ilj*x). 

See Appendix [D| for a proof of Theorem 3.2 


It is worth mentioning that (1,1, 0o) is an element of 5 a for every meaningful choice of A since otherwise 
the optimal strategy would be not to take any samples at all, but to decide on a hypothesis a priori, cf. 
Novikov| ( 2009| ). Such a trivial strategy can indeed be cost minimizing, if A is chosen sufficiently small. 
However, by only considering target error probabilities from the open unit interval, i.e., 7 i G (0,1), trivial 
tests are excluded from the set of possible solutions of the constrained problem. 

Theorem|3.2|is central to this work since it directly connects the solution of the optimal stopping problem 


p.4| ) to the error probabilities of the corresponding sequential test. Exploiting this connection is key to 
converting the sequential testing problem ( |2.1[ ) into a linear program. 

4. Sequential Testing as a Linear Program 

In this section we derive, based on the results in Sections[^and[^ the linear form of the sequential hypothesis 
testing problem stated in ( |2.1| ). 

Similar to the approach in Section]^ the first step to a linear program is to include the constraints in ( 2.1 1 
in the cost function. However, the cost coefficients here are not chosen a priori, but are instead introduced 
by directly applying the machinery of Lagrangian duality to the problem. 

The Lagrangian dual problem of p.l|) is given by 


max L.v(A), 
A>o ' 


(4.1) 


where 


L^(A) = min {E[T{'fi)] + Ao(ao(V’, </>) - 7o) + Ai(ai(V', fi) - 7 i)} 

P,4> 

= min{£’[r(V^)] + Xoao{'fi,f) + Xiaiif, fi)} - Ao7o - A 171 

P,4> 

= mmVxitp) - Ao 7 o - A171 
P 

= Vxir) - Ao7o - Ai7i 

= f’A(l, 1, ^ 0 ) — Ao7o — Ai7i. 
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is concave in A by construction. However, the equivalence between ( |4.1| ) and p.l[ ), i.e., the absence of 
a duality gap, is not obvious. It is therefore stated explicitly in the following theorem, which is mostly a 
corollary of the results stated in the previous sections. 

Theorem 4.1. Let A* be the solution of It then holds that 


= and = E[Tif;)], 

i.e., the solution of coincides with the solution of 
See Appendix]^ for a proof. 

By ( |4.1| ) and Theorem|4.1[ the original problem ( |2.1| ) is equivalent to the maximization problem 


max pa( 1 , 1 , Oq) - Ao7o - A 171 
A>0 

s.t. px{z,e) = m.ui\gx{z) , 1+ pxdHz^e 


(4.2) 


The simple trick at this point is to relax the equality constraint to an inequality and add px to the set of free 
variables. It yields the main Theorem of this work. 

Theorem 4.2. Let L be the set of nonnegative Hzfi integrable functions on E. The problem 

max p{l, 1 ,6»o) - A070 - A171 ( 4 . 3 ) 

A>0, 

s.t. p{z,6) <mmi^Xozo, XiZi, 1 +j pdHz^e^ M{z,6) £ E 
is equivalent to problem More precisely, 

=P*(1,1,^o)-Ao70-Ai7i, 
where X* and p* solve Q. 


Proof. Qualitatively speaking, the validity of the relaxation in ( |4.3| ) follows from the fact that every p{z, 9) is 
a nondecreasing function in p{B) for all B £ £. Therefore, maximizing p at one point implies maximizing 
p over the entire state space. 


To formalize this, let p* be the solution of ( |4.2| ) and p be the solution of the corresponding relaxed 
problem ( |4.3| ). Since p* is unique, p = p* whenever p fulfills the relaxed constraint with equality. Hence, 
only the case when equality does not hold needs closer inspection. In this case, a function p + Ap with 


Ap = min 



1 + 



p > 0 


can be constructed, without changing A*, that still fulfills fhe inequalify consfrainf, buf dominafes p. This 
procedure can be repealed lo create a nondecreasing sequence of funclions lhal converges fo a solulion of 
fhe non-relaxed problem ( |4.2| ). Since p is assumed lo be optimal, Ihis means lhal p < p*, bul /5(1,1, 9q) = 
p*(l, 1, 0o)- For this lo hold, p and p* musl differ only on a P null set The associated slopping rules are 
hence equivalenl in an almosl sure sense. □ 
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4.1. Discussion 


Problem ( |4.3[ ) can be written as a generic linear program by splitting the minimum-constraint into three 
linear inequality constraints. However, since it involves the optimization over a continuous function, it falls 
in the class of infinite-dimensional optimization problems. The solution methods for this kind of problem 
range from classic calculus of variations ( Gelfand[ [2003 1 ) to general numerical approaches ( |Schochetman[ 
2001} Devolder 20101 and approaches customized for linear problems (|Ito[ 20091. However, a detailed 


analysis of infinite-dimensional optimization techniques is beyond the scope of this work. For the examples 
presented in Sectiona straightforward discretization of the problem proved sufficient. 

The result of the optimization are optimal cost coefficients A* and the corresponding cost functions gx* 
and px*. The maximum value of the objective function, i.e., px*{^, 1, Go) — Ag 7 o — Ajyi, corresponds to 
the expected number of samples of the test. The optimal stopping rule is to continue the test as long as 
Px* {z^, On) < gx* {z'^) and to stop if px* {z^, On) = gx* (z'^)- Calculating the boundary dSx explicitly is in 
general not necessary, but can be done to reduce the amount of storage and to compare the optimal test to 
constant threshold tests — see Section |5] 

When solving problem ( |4.3[ ) numerically, it can be the case that on some region B C E the inequality 
constraint is not fulfilled wifh equably, even Ihough B is nol a P null sef. This effecl is due fo numerical 
inaccuracies and occurs when fhe coupling befween p{l, 1, 0o) and p{B) is so weak fhaf fhe confribufion 
of B lo p{l, 1, 6q) is smaller lhan fhe precision of fhe solver. As a resull, fhe slopping region can exhibif 
some areas, where fhe cosl for conlinuing is erroneously declared lo be smaller lhan lhal for slopping. 
However, given a reasonable precise solver, Ihese arlifacls occur only in regions of fhe slafe space lhal are 
highly unlikely lo ever be reached during a fesl and usually are a purely cosmefic problem. In any case, fhe 
procedure given in fhe proof of Theorem |4.2| can be used lo conslrucl a valid solulion from fhe inaccurate one. 
Allernalively, a regularizalion term can be added lo Ihe maximizalion lhal explicilly enforces equably — see 
Appendix]^ for delails. 

An advanlage of fhe LP design approach is lhal il does nol require addilional performance analysis. The 
expected run-lenglh is already a resull of Ihe oplimizalion and Ihe required error probabililies are mel exaclly, 
in Iheory, or wilhin Ihe accuracy of Ihe numerical solver, in praclice. This is in conlrasl lo many design 
techniques lhal are Iwo-slep procedures: Firsl, a slopping rule or Ihreshold is determined, based on upper 
bounds on Ihe error probabililies. Second, Ihe performance of a lesl using Ihis slopping rule is analyzed. 
While such approaches work well in Ihe i.i.d. case or Ihe asymplolic case, where limiling dislribulions and 
large sample number approximalions can be used, Ihey become increasingly involved and inaccurate for Ihe 
case of correlated observalions or moderate largel error probabililies. Hence, Ihe design of sequenlial lesls 
under such scenarios often relies on Monte Carlo simulalions ( Tarlakovsky [ [2003 1 or resampling melhods 
( Sochmanj 20051 lo determine Ihe Irue error probabililies. 

The use of classic numerical oplimizalion melhods for Ihe design of sequenlial lesls is uncommon, 
perhaps because of Ihe hidden nalure of Ihe linearily. The problem is indeed highly nonlinear in Ihe obvious 
oplimizalion parameters such as Ihe slopping rule or Ihe likelihood ralio Ihresholds. More widespread is 
Ihe praclice of evalualing Ihe performance of a given slopping rule by solving Ihe Fredholm equalions ( |D.1| ) 
numerically ( Tarlakovsky[ 20141. However, for Ihe purpose of designing sequenlial lesls, Ihis approach has 
ils limils since a reasonable eslimale of Ihe slopping region has lo be known beforehand. While Ihis is 
unproblemalic in Ihe i.i.d. case, conslrucling slopping regions by hand becomes exlremely challenging for 
Ihe case of correlated observalions. The relalive simplicily of Ihe linear programming approach, in conlrasl, 
is achieved by avoiding a direcl calculalion of Ihe slopping region all logelher. In Ihis sense, il is remarkable 
lhal Ihe boundary manifolds lhal solve Ihe Fredholm integral equalions in ( |D.1[ ) can be oblained implicilly 
by solving ( |4.3[ ), whereas performing an explicil oplimizalion wilh respecl lo dS is a formidable lask. 
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5. Examples and Numerical Results 


Three example problems are solved in this section to illustrate the proposed LP approach to sequential 
detection. The basic task in all of them is to test for a shift in the mean of a Gaussian random variable. 
The dependency structures, however, are chosen increasingly complex. A simple i.i.d. model is considered 
first, followed by an observable Markov chain with two states. Finally, the sequential testing problem for 
the Gaussian AR(1) process is solved. To the best of our knowledge, the optimal solutions for the two latter 
models have not been given in the literature before. 

The sequential tests in this section are all designed to minimize the run-length under the null hypothesis. 
The main reason for this choice is that zq = 1 for P = Pq so that the optimal test can be performed on zi 
only. This significantly simplifies the plots and reduces the technical difficulties. In addition, minimizing 
the run-length under a particular hypotheses is a task often encountered in practical problems, like hazard or 
fault detection. For notational convenience 2 is used instead of zi. 

The only input to the linear program that is not chosen by the test designer is the family of measures 
Hzfi. In the following it is represented by a kernel function h{z', O '; z, 9) satisfying 



h{z',e'] z,e)dz'de' 


VPEf. 


Note that determining h is the only non-generic step in the test design and therefore the most likely source 
of errors. 

In order to numerically solve problem ( |4.3| ) all continuous quantities are discretized, including the kernel 
of the integral transformation. This corresponds to solving the problem on a grid with a finite number, M, 
of points. The discretized problem in its generic form reads 


max Pn-Ao 7 o-Ai 7 i s.t. p < min {Aq^o , AiZi , 1-f piT} , (5.1) 

where p, z are row vectors of size M and H is a matrix of size M x M that corresponds to the integral 
kernel. The index n is chosen such that p„ corresponds to p(l, 1, Oq). For the problems presented here, this 
straightforward sampling approach is sufficiently accurate and computationally efficient. 

For better numerical stability, however, it is highly recommendable to perform some kind of pre-warping 
or to use a nonlinear sampling function since the likelihood ratio values require different sampling granular¬ 
ities on different intervals. For our experiments, we used 

where t : M+ —>• (0,1] maps the positive reals onto the unit interval and /3 > 0 can be chosen freely. This 
mapping can be interpreted as the concatenation of a logarithmic transform and a logistic transform. 

The results in this section are presented in terms of log-likelihood ratio thresholds. Following the ma¬ 
jority of the literature, the upper threshold is denoted A and the lower threshold B. In general, the optimal 
thresholds are functions of the past observations, i.e., A = A{9) and B = B{6). For the sake of a more 
compact notation, the cost function associated with continuing the test is denoted 


dx{z,6) ■=l + j pxdH^^g. 

The equivalent cost function for stopping the test is gx{z). 

As a reference for comparison with the optimal results, tests are used whose thresholds are calculated 
according to Wald’s approximation, which is given by 

A ga log , Ssalog—(5.2) 

1 — ao i — ai 
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These approximations are independent of the distributions underpinning the test and are most widely used in 
practice—irrespective of the fact that improvements have been suggested throughout the years ( Page[ 1954| 
Tallis[[T965| ). It is further shown in Wald (19481 that the approximations ( |5.2| ) are asymptotically optimal as 
max{7o,7i} 0 . 

Finally, in this section the quantities marked with a tilde have been obtained by means of Monte Carlo 
simulations. In all of the experiments, 10® Monte Carlo runs of the respective sequential tests were per¬ 
formed. 


5.1, Mean Shifted Gaussian IID 


The classic problem of “testing that the mean of a normal distribution with known standard deviation falls 
short of a given value” ( Wald[ 19471 is a good example to introduce the LP approach to the design of 
sequential tests. It can equivalently be formulated as 


Tfo : Xn ^ fjv{Xn] 0,cr), 

Hi: Xn ^ fjvixn] 


where ; //, a) denotes the Gaussian probability density function with mean /i and standard deviation a. 
All Xn are assumed to be i.i.d. under both hypotheses. 

Since every observation is independent of the past, Eq = 0. Under P = Pq, the log-likelihood ratio 
s = log(z) follows a Gaussian distribution with fig = and ag = fJ^/cr such that the kernel in 

terms of s is given by 


hfidis', s; 1 ^, a) = fj^{s' -s] ^g, dg) = 


a 




exp ( -^(s' - s) - 




(5.3) 


To obtain an expression in z, one can simply substitute s = log( 2 ;) in ( |5.3| ). Alternatively, problem ( |4.3| ) 
can be formulated directly in terms of the log-likelihood ratio as 


max p( 0 , 0 , 9o) - Ao 7 o - A 171 
A>0, p>0 


s.t. p{s, 9) < min < AoC^” , Aie^^ ,1+1 pdHgQ 


Expressions in tp{z), or any other bijectively transformed version of z, can be analogously stated. 

For the experiments the parameters p = a = 1 and tQ,^{z) was sampled at 200 equally spaced points. 
The kernel matrix iT-jj is accordingly of dimensions 200 x 200. Problems of this size are solved within 
seconds by state-of-the-art LP solvers, which makes the algorithm very attractive for the design of sequential 
tests between two i.i.d. sequences. 

From Table and it can be seen that the optimal sequential test can perform significantly better than 
the test using Wald’s approximations. In particular, in cases where large overshoots over the threshold can 
be expected, i.e., for large error probabilities, the average run-length is reduced by up to 25%. For smaller 
error probabilities the improvement is less pronounced, as was expected. 

How the optimal thresholds can be obtained from the results of the LP problem is illustrated in Figure [T] 
Here the costs for stopping and continuing the test are plotted as functions of the likelihood ratio. The points 
of intersection correspond to the thresholds. It is noteworthy that even if the target error probabilities are 
chosen to be identical, the values of the optimal cost coefficients differ significantly. This difference can 
be explained as follows: Since the run-length is minimized under the null hypothesis, the likelihood ratio 
sequence admits a permanent drift towards the lower threshold. Choosing the latter closer to zero signifi¬ 
cantly reduces the run-length at the cost of an increased probability of second type errors. The probability 
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70/71 

-^opt/ -®opt 

5^0,opt/5:i,opt 

.^olTDpt]/ [Topt] 

[Tapt] 

0.1 

±1.62 

0.0995 / 0.0996 

3.77 / 3.78 

3.78 

0.05 

±2.36 

0.0505 / 0.0497 

5.57 / 5.57 

5.58 

0.01 

±4.03 

0.0097 / 0.0099 

9.31 / 9.29 

9.28 

0.1 / 0.01 

1.70 / -3.93 

0.1012 / 0.0097 

7.93 / 4.69 

7.91 


Table 1 . Mean Shifted Gaussian IID: Optimal log-likelihood ratio thresholds A, B, empirical error proba¬ 
bilities cti, and average and expected run-length r for target error probabilities 7 


70/71 

^Wald/ 7?Wald 

cio,Wald/dl,Wald 

Eq [t Wald] /El [t Wald] 

efficiency 

0.1 

±2.20 

0.0572 / 0.0578 

5.19 / 5.18 

0.73 

0.05 

±2.94 

0.0286 / 0.0284 

6.94 / 6.93 

0.80 

0.01 

±4.60 

0.0056 / 0.0057 

10.51 / 10.50 

0.89 

0.1 / 0.01 

2.29 / -4.50 

0.0562 / 0.0055 

9.54 / 5.92 

0.83 


Table 2 . Mean Shifted Gaussian IID: Log-likelihood ratio thresholds A, B, empirical error probabilities dj 
and average run-length r for target error probabilities 7 using Wald’s approximations. The last column gives 
the relative loss in the average run-length compared to the optimal test, i.e., i^o['d)pt]/^o['rwaid] 



log 2; 

Figure 1 . Mean Shifted Gaussian IID: Cost functions for optimal tests with error probabilities 7 = 0.1 and 
7 = 0.01. 

of first type errors, by contrast, is mainly determined by the upper threshold, which has very little influence 
on the run-length under Pq. Consequently, first type errors have to be penalized much higher than second 
type errors if both are supposed to occur with the same probability. This asymmetry highlights problems 
with approaches that assume the cost coefficients to be given a priori or simply assume both error types to 
be equally costly. 
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5.2. Observable Markov Chain 


The above example can be complicated by assuming that the observed random sequence is governed by an 
observable Markov chain with state space Eg = {1,2}. More precisely, 


Ho : Xn := {Yn, Qn) ~ Po{0n) ' f^f{yn ] 0, a) 

Hi : Xn := iYn,en) -- PlienK-l) ■ en/2,a), 

where pi defines the (transition) probabilities of the states. In this model, Yn and 0„ are independent 
Gaussian and Bernoulli random variables under the null hypothesis, while under the alternative hypothesis 
the distribution of Yn depends on the current state On- The initial state is assumed to be 6*0 = 1. Apparently, 
6n-i is a sufficient statistic for the distribution of Xn-, conditioned on the previous observations. The integral 
kernel under the null hypothesis can be shown to be a scaled and shifted version of ( |5.3| ), namely 

thuds', O', s,0; a) = po(O') ■ - s ; puc, oTuc), 


where 


PMC = PMcd', 0) 


8^ 


+ log 


Pi{e'\o) 

Po{0') 


and umc = o-mc{0') 


10' 
2 a' 


For the numerical results cr = 1 is assumed and the transition probabilities under Hi are chosen sym¬ 
metrically as p{0' I 0) = 0.8 for O' = 0 and p{0' \ 0) = 0.2 for O' / 0- Under Ho Po(l) = Fo(2) = 0.5 
is used. Again, todd was sampled at 200 points. However, since p\ is now defined on M+ x {1,2}, the 
stacked vector p = {p{z, 1), p{z, 2)) is of size 400 and the matrix x 400. The runtime of 

the solver is not significantly affected by this rise in complexity. 

The results of the optimal test are given in Table and Figure The expected run-length and error 
probabilities of a test using Wald’s approximations are shown in Table The results do not differ much 
from the i.i.d. scenario in terms of the efficiency of Wald’s test. The reduction in samples by using the 
optimal strategy is still between 25% and 10%. However, to achieve this reduction, the likelihood ratio 
alone is no longer a sufficient test statistic since different thresholds have to be used in different states— 
see Figure]^ In line with the asymptotic optimality of Wald’s approximations, the difference between the 
thresholds in the two states reduces with decreasing error probabilities. 


5.3. Gaussian AR(1) Process 


The final example is the Gaussian AR(1) process. In Novikov (20091 it is shown that the optimal stopping 
strategy for this process is a function of the likelihood ratio and the current observation. However, to the 
best of our knowledge, the exact strategy has never been derived, let alone implemented. 

The two hypotheses are given by 


Hq - Xn — O^oXn—l T 

Hi - Xn — (llXn—l Y 6n, 


where {en)n>i is a sequence of i.i.d. zero mean Gaussian random variables with standard deviation a- Since 
knowledge of Xn-i is sufficient to describe the conditional distribution of Xn, On-i = Xn-i is chosen. The 
log-likelihood ratio increment Asn of the single observation Xn is given by 


ASn 


(®1 ®o)^n—1 




(of - al)xl_^ 

2^2 


14 








70/71 

Aopt/ BoptiO — 1) 
Aopt/5opt(0 = 2) 

5 ^ 0 ,opt/Q^l,opt 

-^0 ["^opt] /-^l ["^opt] 

Bq [ropt] 

0.1 

1.76 / -1.47 

1.63 /-1.64 

0.0996 / 0.1005 

3.54 / 4.17 

3.54 

0.05 

2.48 / -2.25 

2.35 / -2.42 

0.0496 / 0.0494 

5.19 / 6.05 

5.22 

0.01 

4.13 / -3.90 

4.00 / -4.07 

0.0097 / 0.0099 

8.69 / 9.85 

8.7 

0.1 / 0.01 

1.85 / -3.80 

1.71 / -3.97 

0.0988 / 0.0096 

7.45 / 5.13 

7.42 


Table 3. Observable Markov Chain: Optimal log-likelihood ratio thresholds A(0), B{6), empirical error 
probabilities dj, and average and expected run-length r for target error probabilities 7 


70/71 

^Wald/ -Bwald 

«0,Wald/«l,Wald 

Bo [t Wald] /Bl [t Wald] 

efficiency 

0.1 

±2.20 

0.0640 / 0.058 

4.84 / 5.51 

0.73 

0.05 

±2.94 

0.0300 / 0.0269 

6.94 / 7.35 

0.80 

0.01 

±4.60 

0.0056 / 0.0055 

9.90 / 11.00 

0.88 

0.1 / 0.01 

2.29 / -4.50 

0.0596 / 0.0051 

9.00 / 6.25 

0.83 


Table 4. Observable Markov Chain: Log-likelihood ratio thresholds A, B, empirical error probabilities dj, 
and average run-length r for target error probabilities 7 using Wald’s approximations. The last column gives 
the relative loss in the average run-length compared to the optimal test, i.e., i^o['dipt]/£'o[Twaid] 



Figure 2. Observable Markov Chain: State dependent cost functions for an optimal test with error probabil¬ 
ities 7 = 0.05. 


The joint distribution of {Sn, Qn), given {sn-i, 9n-i), is nonzero only on the one-dimensional manifold 




S-n Sji—l 
^n —1 ^1 ^0 


+ 


^n—1 


2 


(ai -b ao). 


(5.4) 
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70/71 

5^0,opt/5:i,opt 

-^0 ["^opt] / -^1 ["^opt] 


0.1 

0.0980 / 0.1006 

5.69 / 5.86 

5.64 

0.05 

0.0509 / 0.0477 

7.46 / 6.98 

7.48 

0.01 

0.0095 / 0.0103 

11.25 / 9.06 

11.24 

0.1 / 0.01 

0.0986 / 0.0098 

9.95 / 6.8278 

9.93 


Table 5. Gaussian AR(1) process: Empirical error probabilities Oi and average and expected run-length r 
for target error probabilities 7 


70/71 

A Wald/ .Bwald 

ao,Wald/ai,Wald 

Eo [t Wald] /El [t Wald] 

efficiency 

0.1 

±2.20 

0.0410 / 0.0535 

7.73 / 6.54 

0.74 

0.05 

±2.94 

0.0171 / 0.0253 

9.45 / 7.51 

0.79 

0.01 

±4.60 

0.0027 / 0.0049 

12.98 / 8.97 

0.87 

0.1 / 0.01 

2.29 / -4.50 

0.0366 / 0.0050 

12.33 / 7.0522 

0.81 


Table 6. Gaussian AR(1) process: Log-likelihood ratio thresholds A, B, empirical error probabilities at, 
and average run-length r for target error probabilities 7 using Wald’s approximations. The last column gives 
the relative loss in the average run-length compared to the optimal test, i.e., i^o[A)pt]/^o['rwaid] 


Let the set of Xn that satisfy ( |5.4| ) be denoted X{sn,Sn-i,Xn-i) C 
kernel under Hi, i = 0,1 is 


Using this notation, the integral 


/i* 


ARl 


{s',e',s,e-, ao,ai,a) = 


; ai6,a), 0'£X(s',s,0) 

0 , otherwise. 


Lor the experiment, cr = 1, oq = 0 and oi = 1 are chosen, which corresponds to testing between an AR(1) 
process and Gaussian noise. In contrast to the previous examples, there now are two continuous quantities 
to discretize. Again, 200 sampling points are used for each, i.e., the log-likelihood ratio and the current 
observation Consequently, the vector p is of size 4 • 10^ and IIari of size 4 • 10^ x 4 • 10^. Problems 
of this size can still be handled by state of the art hard- and software, especially since IIari is exceedingly 
sparse, but the limitations of the proposed method start to show. Lor more complex dependency structures, 
more advanced solution methods have to be used. 

The average run-length and the error probabilities of the optimal test and the one using Wald’s approx¬ 
imations are given in Tables and A segment of the cost functions for 7 = 0.05 is depicted in Ligure 
The intersection of the two surfaces corresponds to the thresholds of the test. In Ligure the latter is 
shown together with the approximated constant ones. Interestingly, the optimal thresholds are not uniformly 
tighter than the approximations. Instead, the additional degree of freedom is used to loosen the thresholds 
for observations that are very unlikely under Pq and tighten them in the critical region around the origin. 
Evidently, this strategy is more efficient than uniformly tightening the thresholds. Another noteworthy fact 
is that in contrast to the lower threshold, the upper threshold is far from being constant. This does not con¬ 
tradict the asymptotic optimality of the constant threshold test. It does, however, indicate that there is no 
longer a stopping strategy that concurrently minimizes the run-lengths under both hypotheses, as is the case 
for i.i.d. observations ( Wald[ [194^ |Siegmund[ 19851. Minimizing the run-length under Pi yields a mirrored 
version of the thresholds in Ligure]^ with the lower threshold following the parabolic shape and vice versa. 

A nice property of the optimal thresholds shown here is that they are relatively easy to approximate by 
polynomials or rational functions. In practice, a few coefficients can therefore be sufficient to implement 
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Figure 3. Gaussian AR(1) process: Segment of the cost functions for an optimal test with error probabilities 
7 = 0.05. 



Current Observation x 

Figure 4. Gaussian AR(1) process: Optimal and approximated log-likelihood ratio thresholds as functions 
of the current observation Xn for target error probabilities 7 = 0.05. 


a nearly optimal strategy that combines the ease of the constant threshold test with the efficiency of the 
optimal one. 
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In addition to the results given in Table an optimal test for the AR(1) model was designed with 7 = 
(0.0410, 0.0535), which are the (empirical) error probabilities of the Wald test with target error probabilities 
7 = 0.1. The idea is to compare the strictly optimal test to the optimal constant threshold test. The 
expected run-length of the optimal test is 7.45, compared to 7.73 for the test with constant thresholds. This 
corresponds to a reduction of about 3.6%. Whether this improvement is worth the increased complexity 
surely depends on the actual application. However, calculating the optimal constant thresholds is a non¬ 
trivial problem in itself so that the effort might as well be invested in solving the problem exactly. 


A. Proof of Theorem 2.1 


Theorem |2.1 1 is a corollary of fundamental results in optimal stopping theory and variations of it have been 
proved repeatedly in the literature ( Shiryaev] 1978| Novikov[ 20091. The proof presented here does not 
differ significantly, but is included for the sake of completeness and to introduce concepts and notations that 
are used throughout the paper. 

Under the usual assumption that decisions are not allowed to depend on future observations, the minimal 
cost V* of a stopping problem is given by the limit 


U* = lim Vo,N, 
N^oo 


where for each N > 1 the sequence (14,Af)n>o is recursively defined by 




wifh basis = C]\[,n —see Shiryaev ( 1978| l; Peskir (20061; Poor ( |2009 1. Here N denofes fhe finife time 
horizon of fhe fruncafed stopping rul^^c„_Ar fhe cosf for sfopping af fime n given horizon N, and Vn^N the 
cosf for sfopping af fhe opfimal fime insfanf befween n and N. 


For fhe sequenfial defecfion problem, Cn,N is obfained from \2A\ and is given by 


Cn,N = n + gx{z^) 

for all N > 1. Assuming fhaf fhe optimal cosf function is of fhe form Vn,N = n + On) for some 

n < N,ii can be shown, via induction, fhaf 


Vn-i,N = min{(n - 1) + gx{z^ ^), E[Vn,N\y^n-i]] 

= min|(n - 1) -f gx{z ^~^), n -f T;[p”’^(2:"-, 6»„)|JT„_i]| 

= (n - 1) -h min ^gx{z ^~^), 1 -h , 6l„)| 

= (n - 1) -h {z'^~^,en-i), 


where 


E[p';:^{z^,en)\En-i] 




/o,0n-i 

/ Qn-l 


z 


n—1 

1 


fSn-l 



dEf 


dn-l 


is a function of z" ^ and On-i- The induction basis is given by = cn,n = N + gx{z^) so fhaf 

= gx- Hence, fhe minimum cosf of fhe A-fruncafed fesf is Vo,7V = P^x^Po) = PA^(i-) ^o) for 

all N> 1. 


* A stopping rule is said to be truncated, if it is guaranteed to stop after at most N time instances, i.e., if an integer > 1 exists 
such that P{t < N) = 1. 
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To determine the limit = limjv->.oo P\^ ’ iiote that is obtained from by applying the 

transformation 

T {p{z,e)} = m.\B.^gx , 1 + j ^ 

which is monotonic in p. Since p^’^ = gx, independent of N, p^^ can be expressed as 

p'^xN =TN-{gx}, 


where T"^ denotes an n-times repeated application of T. The transition to the non-truncated case yields 


lim p"’ = lim T {gx} = lim {gx} =■ px 

N^OO N^OO N—¥oo 

for all n G N. To show that limAr^oo {pa} exists and is unique, it suffices to show that T"^ {pa} > 0 
for all n > 1 and that the sequence {5 a} is monotonically nonincreasing (Rudin 1987 Novikov[|2009 1. 
The fact that (pa) > 0 follows directly from 5 a ^ 0 and the definition of T. The monotonic property 
can again be established by induction. Assume that {pa} < {5a}- By monotonicity of T it then 

holds that 


r- {pa} = T {r-H5A}} < T{T^{gx}} = {5a} . 

The induction basis T {5 a} < 5a holds trivially since T {p} < 5 a by definition. The fixed-point solution of 

P = T{p} 


yields ( |2.4| ). This concludes the proof. 


B. Proof of Lemma |3^ 


Uniform convergence of monotonic sequences often follows immediately from Dini’s theorem (Rudin 


19871. In this case, however, neither is the state space E compact, nor is px necessarily continuous in 
9. Nevertheless, uniform convergence can still be shown via a detour over almost uniform convergence. 
Define fhe measure 

H*{B)= sup Be£. 

(z,e)£E 

By Theorem |2.1| {px)n>o converges poinfwise on E and hence H* almost everywhere. Egorov’s theorem 
(Beals 2010l states that this implies almost uniform convergence with respect to H*, i.e., for every e > 0, 
there exists a set £ £ such that H*{B^) < e and {p^)n>o converges uniformly on 77 \ B^. In the 
following it is shown that for px almost uniform convergence implies uniform convergence. 

Since {p'x)n'>o is monotonically nonincreasing, p” can be written as p” = 5 a + £^P^ for every n > 0, 
where (Ap”)n,>o is a nonincreasing sequence of nonnegative functions. To guarantee uniform convergence 
it suffices fo show fhaf limn_>.oo sup(^ Ap" = 0. By definifion of p” if holds fhaf 

sup Ap^ = sup \mm.\gx{z),l+lpl~^d.H^A-px{z,9)\ 

{z,e)&E {z,e)&E I I J J J 

Slip |min|5A(2;), ^ + j 5a - pA + j Ap^'MiT^^ej 


= sup 7 / Ap” dH^^e 

{zfi)&E 
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With defined as above, it further follows that 

sup Ap" < sup I / 

(z,e)&E {zfi)&E U 

< [ sup Apl-^dH*+ [ sup Ap^-^dH* 

Je\Bs {z,e)eE\Bs JBe iz,e)eB^ 

< sup Apl~^+e sup Ap1~^. 

{z,e)£E\B, {z,9)£E 

Since lim„_).oo sup('2^6i)g£\B^ Ap” = 0, the sequence (sup(^ Ap^)n>o converges to zero for every 
e < 1, given that sup(2,6))eE bounded for some n. The latter is guaranteed by the pointwise conver¬ 

gence of p” on E. 


C. 


Proof of Theorem 



The proof of Theorem |3.1| is given in two parts. First, it is assumed that px is differentiable almost every¬ 
where and, in particular, allows for interchanging the order of integration and differentiation. In the second 
part, these assumptions are refined and justified. 

Assuming differentiability, taking the derivative with respect to A* on both sides of ( |2.6| ) yields 


p'\iiz,0) 


g'xS^), for{z,e)€Sx 

^ d f 

— I pxdH^^e, for {z,e) € Sx- 


On dSx the derivative is not defined. Assuming, for now, that 

PX dH.^e = I Px, dH,,e (C.l) 

it follows that 


P\ii^,G) = 9xiz)ls^ + 


= 9xiz)'i-S^ + 




(C.2) 


On Sx the derivative of gx exists everywhere, except on {2;o-^o = which is a null set by Assumption 

3, so that 


9'xi = 

Substituted into ( |C.2| ) yields 

= Zilsx, d- \Zi ^ ^ ^ dFi^e + [ px-dHzp \ 




/— ' 
'Sx 


— zdsxi ( ^'^z,ei'Sx,) + J_ p'xi dHzfi ) 


which is the statement in Theorem l3.ll 


(C.3) 
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At this point, it still needs to be shown that p\ is differentiable on S, that the order of integration and 
differentiation in (|C.1|) can indeed be interchanged and that the integrals in (C.31 have a unique solution. 


First, the question of differentiation under the integral is addressed. By the derivative lemma (Bauer 
2001] ), ( |C.1| ) holds, if 


1. Pa is Hzfi integrable for all (z, 0) ^ E (already shown) 

2 . Pa is differentiable almost everywhere 

3. an integrable function f that is independent of A exists with r - I^aJ- 

Properties 2 and 3 can again be shown by induction. Consider the three sequences of functions (i?")n>o 
and (r” )„>o, z = 0,1, with i?” : E M+ recursively defined by 


Rx = 9xiz)ls^ + (^J 




and : Sx 


by 


rl{z,9) = ZiHl,{SxJ+ _ 

JSx 


The induction bases are given by = gx and = Zi, respectively. Following the same line of arguments 


as in the proofs of Theorem 2.1 and Lemma 3.1 it can be shown that {R^)n>o converges monotonically and 


uniformly to a unique, nonnegative function Rx- Analogously, (r” )„>o converges monotonically and uni¬ 
formly to the nonnegative function ta-. From the uniqueness of px and Rx, it further follows that Rx = px- 
Now, assume that R^ fulfills fhe differenlialion lemma and fhaf r", {z, 6) is ifs derivative wifh respecf fo 
Aj on 5 a. It then holds that on 5 a, 


d 


5T-A*'= / I = 


d 


_ 


dX, 


meaning that j^Rx^^ is well defined on Sx and is upper bounded by r = = Zi. Consequenfly, R^ 


dX 


pfl+l 


again fulfills fhe differenfiafion lemma. The facl fhaf on 5 ^R-x = ^Sx = Zi = complefes fhe 
induction. 

In summary: The sequence (i?^)„>o converges uniformly fo px on E. The sequences of derivafives 
{r^)n>o converge uniformly fo rx^ on Sx- From fhis if follows (Rudin[ 19871 fhaf px is differentiable on Sx 


wifh p'^ = ta- < Tj = Zi, which justifies fhe use of fhe derivative lemma in ( |C.1[ ). 

D. Proof of Theorem!: 


3.2 


The error probabilifies ai are given by 

ai = Pi[(f>r = I - i] = Pi[(j)T = 'i-- i \ Z^ = 1, 0° = ^o]. 

Since fhe underpinning Markov process is time homogeneous and fhe opfimal slopping rule time invarianf, 
if further holds fhaf for all n, m > 0 

Pi[(j)r = ^ — 'll z"' = z, 9^ = 6, T > n] = Pi[4>r = ^ — i \ z"^ = z, 9^ = 9 ,t > m]. 

It is therefore possible to define fhe funcfion 

ai{z, 9) = Pi[(l)T = 1 — i \ z'^ = z,9'^ = 9 ,t > n], 
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which is independent of re. From the Chapman-Kolmogorov backward equations (Peskir 20061 and the 
definition of FT* g in ( |2.7| ) it follows that 


ai{z,9) = J aidHlg. 

For -i/; = -0^ the stopping region is given by Sx- Choosing (j) = (j)*^ further yields 

afiSxi) = 1 and a'l{Sx,_,) = 0 

for all re > 0. Hence, 

a,iz,e) = Hln{Sx,)+ [ aidHlg 


(D.l) 


on 5 a. These are the well established Fredholm integral equations describing the error probabilities of 
time homogeneous sequential tests (Feller 1971t Tartakovs^i] 20141. Note the difference in the integration 
measure compared to ( |3.1| ). Using the same techniques as before, it can be shown that ( |D. 1| ) has unique, 
positive solutions on 5 a. What is left to show is that these solutions coincide with p'x !Zi. 

From TheoremB.llit is clear that on Sx 

P'\i Zi 

— = — = 1 = Ui 


and on 5Ai_i 


—L=o = ai. 
Zi 


On Sx it needs to be shown that p'x JZi solves ( D.l i, i.e.. 




Zi 


= + 


P\, 1,^ 0 fe fg 

■y.tiA 
fe 






p'xS^^P) 


Zi 


= HleiSx^) + - ) dF, 


Js 


fo,e fi,8 


p'x^iz,9) = z,Hlg{Sx^)+ _ p'xAH.^e, 

JSx 


which is true by Theorem 3.1 


E. Proof of Theorem 4.1 


Assuming that attains its maximum for some finite and positive A*, and that its derivatives in this point 
exist, it holds that 




a=a; 


= 0, f = 0,1. 


By Theorems 3.1 and 3.2 


d d d 

—L^{\) = —pa(1, 1, 9q) - ^(Ao7o + Ai7i) 

= Pa,( 1 i 1 >^o) -7i 
= ai('/>A>V'A) -7i 


(E.l) 
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so that for A = A* 


= li- 


The test therefore meets the target error probabilities exactly and is of minimum expected run-length by 
definition of p\. The dual objective accordingly is 


L^(A;) = pa;(1,1,0o) - a^oto - a;,i7i 
= ^a;(V^I;) - A;o7o - A;,i7i 

= E[Tirx*)] + A;o«o(</>I;,V-I*) + X;pai{<Pl,,rx,) - A^oTo - A;i7i 
= E[T{rx*)] = E[T{r^)]. 

Finally, it needs to be shown that A* is indeed positive and finite. First, let some A* ^ = 0. This implies 
px = 0, which corresponds to a trivial test—see the remark at the end SectionSince in this case Ui = 1, 
it follows from ( |E.1| ) that 

a\* = 1 — 7 i > 0 , 

which contradicts the assumption that A* maximizes L^. Second, increasing some Aj indefinitely leads to a 
test with Oj —)■ 0 so that 

lim J^Ly(A) = 0 - 7 i < 0. 

Aj—^oo Cy/\j^ 

Therefore, Ly{X) is decreasing in the limit and, in turn. A* is bounded. 


F. Enforcing Equality in the Constraint of the Relaxed Linear Program 

If numerical problems arise in the solution of ( |4.3| ), such that the inequality constraint is not fulfilled wifh 
equably, a regularizalion lerm can be added lo Ihe objeclive funclion, namely. 


max p(l,l,6»o) - Ao 7 o - Ai 7 i+ c pdp 
X>0,p£C J 

s.f. p{z,9) <mml^XoZo, XiZi, 1 +j pdH:,^ 0 ^ \/{z,e) e E, 


(F.l) 


where i] is some sfricfly increasing measure on {E,S) and c is a small posifive consfanf. In ( |F.1| ) p is 
explicifly maximized over fhe enfire slate space, whereas in ( |4.3| ) Ihis maximizalion resulled indireclly from 
maximizing /?(!, 1, 0o)- Note fhal Ihis regularizalion of fhe original problem is by no means fhe only way 
lo combaf numerical arlifacls and is nol essenlial fo Ihis work. Neverlheless, if is slraighlforward and yields 
good resulls in practice. 

Since p is increasing in A, c has fo be chosen small enough such lhaf fhe problem is still bounded. To 
guaranfee Ihis, fhe addifional infegral term can be upper bounded by 


J p dp < 


j gdp < Xo 


zo dp + Ao 


d?7 = Ao + Ai, 


where, wilhoul loss of generalily, if is assumed fhal p has been chosen such fhal 


j Zi dp = I, z = 0,1 

The regularized objeclive function in ( |F.1| ) is Ihen bounded by 

p(l, 1 , 00 ) - Ao(7o - c) - Ai( 7 i - c). 
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Consequently, ehoosing c < min{7o,7i} guarantees boundedness. Furthermore, this shows that the reg¬ 
ularized problem eorresponds to the original problem with smaller target error probabilities, which means 
that the solution, even though not strictly optimal anymore, still satisfies the original error requirements. 

For the discretized problem, the additional integral term can be replaced by a weighted sum of all 
elements of the vector p and the above considerations can be used to determine the constant c so that the 
solution of the regularized problem is sufficiently close to the original one. 
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